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• . Preface, 

The development work reported here was the first in a planned , series of* 
steps aimed at evaluating — in performance terms--certain outcomes of the 
University of Puerto Rico*s special* Curso de Perfeccionamiento (an offering 
provided for unlicensed graduates of foreign medical schools). The intent of 
the projected evaluation scheme was to assess the Curso's effects on the *'qual- 
ity of medical care and on community health'* as engaged in by those Curso 
graduates who later approved the state licensure examinations and took on vari- 



ous assignments around the Island. 

As it Aumed out mounting such an evaluation plan proved costly beyond 
the means /of the existing grant; moreover, it also became clear that the Curso 
graduate^ were- entering a much wider range of medical practice (and study) 
opportunities than had originally been anticipated. It was not possible to 
continue this particular line of evaluation work (although several other as- 
sessm^ts of the Curso were undertaken and have been reported separately). 

le staff of the Curso and other members of the 'medical^ community saw 

valu)l, however, in the initial step of defining adequate ^care immeasurable 

terms. It was felt that the specification of such criteria — ^^even if not 

/ < ■ > ♦ 

refined to the ultimate form needed in -an evaluation of p^jrf ormance--would 

hdve a number of potential applicatipns in Puerto Rico. These are discussed 

in the body of the report. 

The project was initiated in the Fall of €970, and the work of revision 
continued for some time after that. Three primary sorts of **gui dance" were 
employed during this period: (a), a review of the literature on definition and 
measurement ^Df quality medical care (available as ET^ Project Report 73-23); 
(b) the^ experience and expertise of a number of medical personnel in Puerto 
Rico who prepared the initial topics and criteria, and somp of whom later're- 
viewed and refiiTe^ the p'roject products ;• and ''(c) the assistance provided by 
ETS professional staff ih matters related to criterion-specification and measure- 
ment requirements. . The ETS ''function" was to facilitate the operational 
statement of criteria, to aid in the .definition of quality care dimensions, 
to record the committee deliberations, apd to prepare the guidelines *in written 
form broth before and after committee review. The major work, of course, was 
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medical in nature, and it was the Puerto Rican medical personnel involved 
who directed the conference and outlined its scope, who determined the areas, 
topics, and specifications needed, and who ultimately judged .the adequacy of 
the criteria thus formulated. 

As noted by one of the participants, the process involved was 'probably 
as valuable as the product. For this reason, several versions of th^ criteria 
are inMuded in this report--to indicate something of the historical develop- 
ment involved and the changing assumptions that underlay the revisions made. 
In addition, of course, tfie very fact of such a conference and the dedica- 
tion of its members in cooperatively attacking the criterion problem^ is a 
notable aspect of the process. Finally, inclusion, of the several successive 
versions may enhanV:e the potential applications of the quality-care criteria. 
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INTRODUCTION AND PURPOSE 

In 1969, the Medical School of the sUniversity of Puerto Ric^o (UPR) re- 
ceived a grant from the National Center for Health Services Research and 
Development (Public Health Service, HEW) to plan and conduct--as a demonstra- 
tion** project--a special retraining program for physicians who had failed to 
approve the licensure examinations * of the Puerto Rico Bo|rd of Medical Ex- 
aminers. These physicians, graduates of foreign medical schools, were iden- " 
tified and -subsequently given^an opportunity to enroll in a six-month "Curso 
de Perfeccionamientp" which included supervised clinical practice along with 
lectures and seminars in the basic and clinical sciences. At this time, the 
UPR medical staff is *in the midst of the fourth Curso, and thus a total of 
approximately 190 men and women physicians have undertaken this special program. 

A primary goal was to increase the m^pow^r pool available for service 
in the Department of Health, particularly in the local Health, Centers in out- 
lying regions. - A coippanion purpose was to improve the quality of medical 
care* provided and to increase the attention given^o communis and preventive 
?dicine. * . - 

\^ important 'aspect of the*Curso program from the b&ginnii^g was evalua- 
tion. >Aside from the more usual sorts of program and impact assessment, it 
was felt iiighly desirable to determine in some way the' quality of pr^actice 
engaged in after licensure. If this were done, then not only could the pre- 
sumed long-ran^ effects of the Curso be investigated, but also the general 
'•quality of m^dic^l care'* in Puerto Rico could eventually be at least partial- 
ly assessed. 

This document repots on the initial steps taken in that direction, namely 
the^^establisbment^of meaningful performance criteria which would define* an 
acceptable level of medicalNcare provided. It was recognized that any attempt 
to assess the quality of professional perf ormance--or to describe detailed 
parameters of such perfQrmance--\s a sensitive matter. But it was -also rec- 
ognized that^^^ail^qually sensitiveNinatter was the assurance that medical 
-care of a high quality be available iiir Puerto Rico (and elsewhere). 

' ' " \ J 

The*pxoject reported here was conceWed of as essentially a planning 
, ^ step, preceding an^ actual development or aj^ministration of measures in the 
field.' The purppses outlined below ^Iso indicate oertain limitations of 
this effort: 

ER?c : . ■ , ■ i 
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1. to develop an operatiorf^l def initio^ of ^hat constitutes good on- 
the-job medical practice by the individual physician . 



2. to structure that definition multi-dimensionally; that, is, -to posit 
several independent but related areas of activity' rather [than .a 
unidimensional trait of J'quality" ^ 

3. to identify, within those dimensions, the particular attributes ^ 
(knowledges, skills, or ^characteristics) deemed necessary to a 
minimally acceptable practice of medicine 

4. to rel.ate those ^dimensions to the practice of physicians in the 
local Health Centers ^of ^Puerto Rico (whether or not the directors 

n of such centers)- ' . , * 

5^ -to relate those;dimensions and specific criteri^initiaUy to a 

particular population (the licensed graduates of . the special Curso) 
' ^ - ^ . 

6. to treat the total physician role, and thus to delineate aspects 

of /'medical carQ*' within a system'as opposed to "medical practice" 
(which might be construed only as clinical in nature) ' \ 

7. to establish the criterion guidelines necessary for the considera- 
tion of 'appropriate measurement procedures 

PROCEDURE ' ^ . 

The UPR project ^staff selected the vital '"'Competency Committee," its 
general chairman,' and ultimately the three subgroups which would deal wijh 
the criterion question. "The 25 members represented the Regional Medical*' 
Program, the Department of Health, the Board of Medical Examirters, the Sohool 
of Public Health, and. the Medical School faculty (the largest proportion 
belonging to tl^ latter, but spread over a wide ^pectr^nj of departm^ts). 
Appendix* A names all the^ctive |)articipants, including project staff members 
and ETS professional persdtinel involved (which included a -^tatesiSe MD" con- 
sultant) * ' * ' 

\ ^ ^ ' ' * . 

* Of special import ^ce was the creation of the "core committee," or ex-- 

ecutive group, tonsisting of 9 members who became central to both the 

development process and.\the review/ refinement function* 
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Several principal members of the total project met in^.advance to outline 
the agenda and decide on effective grpup. and subgroup strategies. . At that 
time it was decided to approach the LHC physician *s role from three points 
'of view: ambulatory practice, LHC hospital, and community medicine. Other 
advange, preparations included (a) the summary of the relevant research, 
(b) the mailing of a few selected papers to each participant, (c) the gather- 
ing of available descriptive data on the clinics, patient loads, disease 
frequencies, role descriptions, equipment, etc, of representative Local 
Health Centers, (d) delineations of plans and expectations communicateil to 
the Committee members, and (e) duplication of sample materials such as an 
outline of an already-published checklist , for management of a case of myo- 
cardial infarction. 

The three-day conference itself was held in December 1970,. at a site . 
away from the daily busyness of participants. Dr. Luis Miranda, the cc^nfer\^e 
chairman, opened the session with an overview of the Curso and the immediate 
purposes. This Vas followed by brief presentations by three ETS consultants: 
Dr. J. A. Davi"? who spoke on evaluation (see Appendix B for a summary of his 
presentation); Dr. William Kastrinos on the development of performance speci- 
fications; and Dr. Roderick Ir^side on the literature review, dimensipns, * 
and discrete criteri;a. Dr. Miranda, assisted by Dr. Egidio Colon-Rivera,^ ^ 
set the stage *for action by giving the participants their specific ch^ge 
and outlining the anticipated schedule. 

Each of the three subgroups^ (see Appendix A) first determined which 
aspects '6f their given. areas they would include and which omit, under guid- 
ance of- the subgroup chairman. That formidable task accomplished, they then 
concentratied on specific criteria within aspects for the'T^^mainder of the 
conference. At times the subgroups broke inlo smaller committees of 2 or 3, 
-and then reconvened to reith consensus. It goes without saying that the 
discussions were intensive, extensive, and comjrehensj.ve. 

At the end of day one, each subgroup reported its decisions on what 
topics would be covered, and on day three submitted written reports qf the 
detailed outcomes (in terms of. both topics and criteria) to all participants 
for discussion.. 



FRir 



8 



• » * 

And discussion there wasl The final- session resulted in a number of ' 
agreed-upon changes in the output of e^ch subgroup, reinforcing the value «of 
giving the total conference time and opportunity to consider and revise--on 
the spot. It should be added that ori a nujnber of points consensus Was reached, 
but not ^unanimous approval* ^ ' * , ' ^ 

IMMEDIATE OUTCOMES 

The **final reports'* of the conference comprised the output of each sub- 
group* separately : hospital, ambulatory, and community medicine. Because 
they were so.m*ewhat different in nature from one another, and contained certain 
notable segments, they are summarized here-'^even though , a large portion^of 
these immediate outcomes were soon revised considerably' into s singl.e frame- 
vork. • * * / . ^ • 

A/ Hospital Group , ' 

(1.) Developed* record forms for use, in studying aijd evaluating 

record^ (as a check on quality care) on several presenting 

conditions (not diseases) which would presumably warrant • 

hospital admission* Simple indicators of minimum care were 

chosen, and generally followed this outline: 
it'. » • 

a) initial workup and criteria §ot admission 

b) diagnosis and speci.fic pian of care 

c) treatment • and follow-up 

. V d) disposition 

1 

The pctual record forms are extensive and will be found'in a later 
section; Crateria for physician behavior were presented in terms 
of the separate disease-oriented activities listed for each 
-presenting complaint* 

(2.) The 4-part qwtline was further broken into a general outline 

applicable to virtually any disease or disorder, and containing 
17 sequential elements* The idea was that for'any given com- 
plaint, the specific nex:essary features would »be entered into 
this outline, recorded, and judged as essential or not essential. 



When it was applied to^complaints , hov^ever, it appeared 
necessary to have different kinds of judgemental systems (as ' 
will be seen in the actual record forms*^ developed) . ; 

(3.) Listed 'four requirements for the d^elopment of such record 

forms,^ in prder to do the job* The forms must covej and include: 

a) evidence that tlie clinical problem has been identi- 
fied • ' ^ / ' ^ * * 

b) patient risks have beerr^identified 

c) criteria for good in-patient care have been met 

d) if patient referred, evaluation of referral foim 

(4;) Considered the question of the "human and environment factors" 
which affect .physician performance, resulting in a list of 10 
such factors; and adopted sets of criteria and requirements having 
to dD with general standai'ds for Health Center facilities, 
^ conditions, and personnel. 

B. Ambulatory Group 

(1.) Outlined -in great detail the total role of the physician in a 

« • * 'I 

-Health Center, as v/ell.as the working situation, policy and 

other elements. Much of this was topical (regarding, the setting) 

. and s^et no ststndards or requirements; as to physician behaviors 

in the delivery of care, however, the criteria were presented 

in terms of specific quest ions --which carried an i-mplicit 

standard of "yes" answers. 

(2.) The topical outline was as follows: 

I. Concerning the General Setting * ' / 

A. General health policy 

^ resources, supervision, evaluation 

B. Local health policy 

/ statistiqal reports, services offered, clinics ^ 
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C. Facilities and resources 

physical set-up, equipment, services, medical 
^anpower, other personnel . ^ 

II. Concerning the Health Process 

A. Medical records and referral 

• , B. Health center statistical records 

III. Communications 

Physician with health team 
\ active participation, consultation, 
personal relationships 

B. Physician with paramedics 

supervision and feedback (give questionnaires 
to paramedic, personnel) ^ 

C. Physician with patients (questionnaire to patients) 

1. Personal i:elationship 

(e.g., '*Does physician know your name?-'') 

2. Professional relationship (relating to history, 
physical, diag^iosis, treatment ,^^prevent^on, 

♦ rehabi]^itation,c^advi^e> (e.g., ''Did he take 

a history beyond your chief complaint?" ^ "Weire 
you told whether disease was -contagious?") 

D. Physician with base hospital 

administrative, referrals (e.g^. , "Does physician. 

^ , ' include short history and pertinent findings?" 

"Did physician receive feedback from hospital?") / 
* ^ 

E. " Physician with local community. 

(e,g., "Is he widling or available to give medical 
lectures or demonstrations to community groups?") 

The group pointedly noted 'that its outp^at constituted guidelines 
for the selection 'of criteria, /ather than a- set of formulated 
criteria. Thus, vfery few ^actual standards were stated and certain 
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questions were not included (such as admission or unnecessary 
treatment).' 

(4.) As noted in ilI-C-2 above, it was suggested that the patient ^ 
in effect '^evaluate" the physician with respect to aj^wide 
range of physician behaviors and communications^ 

(5.) It was proposed that the outside ev^luator, in studying .records, 
* look for the following; thjese might be done in reviewing all 
records, or in a spot-check, or foi;^ records on a given condition 
in the LHC. 
i 

a) pertinejice and relevance of histoiy, physical, 
as related to chi^^om^laint 

b) agreement of diagnosis, lab, treatment, as 
related to chemical data 

c) ev.idence of proper disposition (ambulatory 

care, hospital, or referral) " ^ 

d) quality of." referral notes 

• e) frequency 'Of^jjfol low -up, as related to illness, • 



severity, 'treatment ' " 

f) interest in personal follow-up 

C, Community Medicine Group 

(1.) Outfined in cpnsiderable detail the total role of the LH^ 

physician re community medicines but within ^ necQssai*y context* . 
of the system and organization overall health care. The. 
^ original catalog of 11 topical areas was reduceyd to '8. 

(2.) .While the committee discussed- the importance of the LHC facility 
and resources, and of the constraints of th^health care system, 
its final report was presented ia^erms o€^he physiciali and his - « 
^ particular responsibilities. E^h topic -was amplified by criterion 
statements (is aware of^ does, examines, keeps, uses, etc.) 
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which In sum gave a very clear picture of. the physici^h«^:S' rble 
' in community health. Thus physician behaviors were dealt With 
dire^tl>j, ixi behavioral terms. • * * " . • '/ 

(3.) The topical outline^ with examples; -was: ' . 

\' I. Legislation ikffecting medical- practice 

l^' r 



:A/ He-l:.ori^i'e3 with existing_laws and regulations 

^(^•g'* practice of abortion, repoxti^'g'of -y^ 
r GOmmuriicabie diseased and cancer). • •• . . 

II. Prevention of communicable ami noncotamunlcable- diseases 

. A. ;He has a. regular immunization program,.. 

B. He takes pecessary measure,s to protect..;- 

C. He actively participates -iiwll groups that '. ^ 
sponsor early-detection programs..^ 

•III. Otganization and utilization of health^sel^vices 

. . • 

A. Refers patients to other medical and paramedical 

personnel at all levels, and t^o other- community^ 
< 

agencies : 

B. He is aware of the' extent of use... 

IV. Group work and relations ' 

A. He works effectively with other members of the 
' health team (e.g., is a source pf ideas, promotes 
and accepts ch'ange) * * ^ 

V. Relatipn between environment and health 

A. Keeps clean and orderly environment at work in- 
eluding water and toilet faci^iities ) 

B. Is* aware of ghy'sical, social, and emotional 
factors that ^ffe(:t the health of* the community^f 

,and takes measures to -correct them where feasible 
(e.g., air and water pollution, family setting)' 
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• VI. Health economics ' 

A. Takes cost into consideration in relation to 
family economic^situation 

, B. Is aware of benefits patients entitled to 
• . (e^g*. Medicare, VA, Social Security} 

VII. Population dynamics 

A. Organizes and plans heallh services .. .in 
relation to population changes 

B. Is aware of overpopulation problems and takes 
V. measures to establish family planning,... ^ 

VIII. 'Keeping^'up with > community needs and professional 

-^pffepar^ti'onr;^^ • - - 

^^ '^-^^s . > r^jAVjCeepsij^iriform^d-Lgf community needs through 
-\surveys, meeting witlT* leaders. ... 

• B. Keeps informed in professional" field. ^ . ^ , 
• . , (e.g., meetings, reading, reviewing-)^: 'I' 

T 

OVERALi: TOPICS AND CRITERIA ' • ' ' 

— * \ 

The products of the conference were next compiled into a single documept 

which combined the concerns and the specifics of the hospital, ajnbula;tt)ry,* 

and community subgroups. The sources used in this compilation included: ^ 

(a) the '*finar* written subgroup 'reports, , (b) notes taken during the delibei^aV 

• , ' - ' • • < \ ' ^ . /y y' 

tions, (c) a tape of the terminal conference session where^ I'eport^ w-eri^ dis- 

cussed and somewhat revised, (d) the Cilrso staff recommendations , and (e.) 

a review by the' ETS medical qonsultant. - : . * • / ^ 

Paramount concern in this document was with the 'dim^nsion^ of -^jifedieal' 

' . ^ . / • -^'. -v: / 
care. The total range of sources was us^d in abstracting th^-^ommon 'dini^ti;- 

sions which 'apfJeared in the three subgroup Veports . After these themes were 

identified, the separate elements of behavior or knowledge' or attitude or 

environment were subsumed, resulting in the elimination of considerable ovef- 

lap and repetition (for example, in the domain of referrals). Another* 
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priority was to^ state ^iie availafiie Standards as Jnuch ais possible in ^er-*;>^ 
formance terms; howevfe'rv^ little li96n.4e was, exercised and the result is 
"uneven''--in the sense that some topics are. xlfetailed into behavioral state- 
ments while/others,merely suggest«a^ atea pf concern. The number of specific 
and precise, indi<;atoTS is small. And a third iiiajor puTOOse was to integrate 
the^Work of the 3 Subgroups to ^ceflect^the real-world situation where the 
individual physician would be worlcipg in hospital, out-patient department, 
and community |iealth. ,cl mics * / , / ' * 

The re6;rganized foiTnulatipn appears on the 'following pages under th^ 
title "Compilation of All Cqpference Outcomes oh Performance CriteVia.'V 
This is divided into, four secjribn?: ^K Dimensions of Medical Car^, .df which 
9 are presented; II ^ Related Dlim^nsio^sVttwo) , concerned wi.th non^-medical, 
topics and criteria; III, £nvitaninehtal' Affecting Quality of .Judge- 
ment and Performance; and IV; RbcordvFoaras o£*Physician\ • 
^ Handling o^) Presenting Complaints^ TTie"'.Battfer were examples tinly, phqsen 
at the Cojvfer^n6e out d-^ d l^rge number o^ possibilities . • / : \. ; - 

, • Tbe^'^phys^-cian wRhq; could cTeal satisfactorily with what is ojut lined here 
^ might iiidee'd b^ rare— especially wlth^ reference to some rural Health Center 
'facJ-iXities' -and resources. The picture that emerges is a. sTomewhat ideal one^; 
moreover: it cl.earl)^ .describes an LHC; dlyedtor in. many of its particulars. 
,{At the tji^e of the Gofeference that' was purposeful, since it ,w§s ai^^^pated 
f that 5ome CursO ;physiciaT% would b^ a)5,signed to small LHC*S where they might 
} h^ve. or. shar6'. the directorv'VcresponsibiMties:* 

: ; A^jnfayj {)e/ apparent from th'ex li^atexial ilb^eli'^^the Conference; members 
con:sidered'/ineasiirem^nt imp^ cations in their w^x:Jc. Study of records wa5 em-/ 
phasize.d, ^a'ltihough observation, inti^^ryiev^, Tatin^s., and patient feedback .were 
also bonsidered'. ^' ' 



^h^iiosoph^icai difference tha;^' does not' show up in the matter ial-.rbut 
Which was the subpect of some dijs'agreement in the Conference--relate^ to the 
validitfy.' of -measurement techniques. . One "school of thought'? held,t*iat the 



A 



discus sdpa 



jdpa.'coTitini 



nued on page 37 I 

; 1 



Compilation /Of All Conference Outcomes on Performance Criteria 
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/% '^e'ctlon 1 : Dimensions of Medical .Gare 

A. CQimnupity> and Femiiy Brotectio^n v > - • ^ 

1. - Keeps Cent-e!r and ofiBice facilities sanitary (including water and 

tojL'let facilities/Xr' * " \ 

2. (a)K^eps informed^of environmental factors that affect cpmmunity 

health, and of community health needs. \ 

(b)Takes measures to cbxrect' these factors where feasible: ^ 
/ y (L) sanjLtary .facilities and conditions 

(2) air and Wajer pollution 
/ ^^(.Sj /family settings and stresses 

, . /' ^' ;X^^ cultural and religious patterns ' ' ^ 
' ' ' :''(5) nutrition, needs 

. , -(6) -community economic situation 
[ (7) oyerpppulation problems- 

- counsels families 

^ -'conducts family planning program 

- refers to existing agencies 

^ (8) communicable disease outbreak (e.g., hepatitis. Influenza) 
^ * - traces source ^ 

- immunizes ' ^ • . * " . 
* • ' . ' - keeps needed medicines on hand 

- follows* patients and contacts 

(9) under- or ovet -utilization of local health services 

3. - Conducts regular immunization programs for smallpox, DTP, polio, 

meq^sles. . * 4 _ 

4. (a)Participates ^n all groups, committees, societies, cl;ubs, that 

work for early detection of heart disease, glaucoma, TB, VD, cancer, 
diabetes. ■ v * 

(b)Coordinates his efforts with these other groups, j 

- (c) Initiates' preventive clinics for these conditions; 

- uses outside personnel 

- works to insure that all who- should attend do attend 
' ' ; - keeps clinic attendance records ^ 

• , - utilizes paramedical personnel in clinic operations 

(d) Encourages* population tp have" regular complete PE*s, 

(e) Initiates and works in prenatal and well-baby clinics. . 

^ ,.(f) Performs* complete PE on susceptible patients or those who have 
symptoms . • . - 

5. - Conducts educational programs tod efforts for individuals, for th6 

community, and for other health care personnel (before and after 
problems erupt), 
t. - communicable disel||gs ^ ^ 

5 - sanitattion 'and pollution 

major nori- communicable diseases 
^ -family planning 

6. - Conducts routine PE*s on school children in the schools. \. 
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B. Participation and Communication Withi n the Tota'l Health System 

1. -'Works effectively with all members of the local health team 

(a) attends meetings and conf eireaces (on time) --of several sorts 

(b) partic'ipates actively in these meetings , : 
M (c) has good interrelationships with colleagues " 

(d) utilizes other physicians for consultations / 

(e) focuses attention on local problems and questions 

(f) is a source, of ideas, proposals, solutions 
' . ^ (g) promotes and accepts cha^ge^ 

. '(h) takes initiative -re local problems . v \ \ 

(i) organizes and utilizes groups and individuals (\ncluding\ 

non-medical persons) in -making certain decisions 
(j) supervises, assists .and gives feedback to paramedical 

• ^ -personnel . ^ 

(k) understands medical roles, and assigns and calls on people 
accordingly (clinics, E,R,, hospital, education) 
\ ^ (1) knows local policies concerning personnel, pharmacy, 

referrals , consultation 

2. - Works effectively with the base hospital, 

(a) k-nows the specialists at base hospital . * , 

(b) knows and uses the correct referral channels 

(c) knows how referrals are handled at base' hospital 

(d) knows how to call pn specialists for either clinic work 

or consultation ^ 



3.(a)Makes appropriate and accurate use^of referral systems. 

(1) refers at local level where needfed ' • y 

- from E.R. to OPD ' ^ ^ 

- to^local private ^ospltal • ^. / ' 

- to appropriate parainedicals '/ " * 
V (2) refers to base (regional) hospital .from E.R. or OPD 

- refers directly by name < f 

- - prepares referral himself (not left tb a .paramedical) 
(3) prepares referrals which: 

- are legible ,\ 

- contain history and physical findings 

- include reasons for referral * ' • 

- include his diagnostic impressions /. 

- show that he attempted to diagnose 

- include any therapy already given 
^ (4) takes patient risks into account 

^ " (5) asks for specific feedback from hospital ^ 

^ /A) ' / - arrival of patient 

- medical report ^ ^ 
^. ^ . - seen by right specialist 

V (6) makes .only necessary referrals ^ • 

(7) refers to appropriate specialist /oi^jOffice 
(b) Keeps adequate records of all referrals and feedback 
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B. Participat jion and Commun-icatiori ^ continued 




4. - Is aware of regulations and recommendations re: ^ 

- patient -doctor ratios * 

- permissible roles of paramedicais ' 
A' - priorities in the use of resources 

./a - reporting of local data to Region and Department 

5. - Maintains all necessary local .records , 

(a) keeps them available 

(b) keeps them organized ^ . , 

(c) keeps them legible 

(d) makes them complete and accurate ! ^ \ 
^ ^ (f) records use of paramedical personjiel' 

(f) records vital statistics ^ • ' 

(g) ' records clinic attendance ^ ^ - 

(h) ^U6es standard forms 

' , ' (i) keeps clinical records on patie,nts 

^ (1) rela^tes history and PE to chief complaint 

(2) ^ diagnosis, lab results, and therapy a|ree 
in' terms of chemical .data 

(3) disposition is made clear ^ ^ 

(4) referral notes are appropriate, complete 

(5) follow-ups are done (depending on severity, 
J therapy, diagnosed illness) 

(j) records local statistics ' 
, >('l) # "patients seen per clinic 

'(2) types of clinics offered 

(3) siirgery perf6rmed and outcomes 

(4) % of pati-ents in E.R. seen by physician on 
' - -3 night call 

(5) diagnostic procedures employed 

(6) medications used for particular illnesses 
^ (7)^^^^^atients ^who already have a medical record 
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Utilization of Mani^ower .Resources in the H.C._an'3^in the co"^^^n|ty 
"l. 



2. 

3. 
4. 



- Provides in.rservice training for paraprofessionals in order'to increase 
Health services available. ' . - ' 

- Maintains^ H.C. morale and sees that employees are punctual, respoji- 

' sible, and stay on job. ^ 

- Gives responsibility to paramedicais as appropriate. '-V;-' 

- Con|ults with and. makes use of other agencies in communil^y as needed 



physicians 

- private clinics or hospitals 

- volunteer groups (aides, education, etc.) - ^ 

- social agencies / - * 

- dentist ' ; 

5. - Attempts to recruit ahd retain adequate staff for^H.C. 

6. - Conducts or participates in (weekly) meetings for sake of communication, 

morale, knowledge of needs,, and best assignments for H.C. staff. 
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D« Utilization of Facilities, Materials, Supplies at the H.C, and in the 
Community 

*^ 

1. - Makes proper use of 

- laboratory facilities and equipment 

- EKG machinery 

- X-ray machinery 
equipment for PE 

- operating room 

2. - Makes sure paraprofe^sionals and nurses can properly use the above. 

3. - Uses medicines -properly. / 

" checks allergies before dispensing - 

; distingui^es. between toxins and^S<wcoids, and among antibiotics 

- does not use*)vhere patient risks ar^involved or where -con- . 

traindicated 

- does not use drugs -which are now considered dangerous because 

of side effects 

- supervises administration of medicines 

t * 

4. - Makes sure equipment stays in working order* 

- diagnostic equipment 

- ambulance 

- immunization ' , ' - ' 

5. - ,Makes sure that supplies are kept on hand* ^ 

- medicines (standard and emergency) 

- medical supplies 

- surgical supplies - * ^ 
^ diagnostic supplies 

6. -.Knows about and makes use of community facilities when needed^ 

(transportation, private hospital', clinics, pharmacies)/ 

7. - Does ;iot hospitalizef those who can be treated in OPD. 

8. - Does not use outmoded or dangerous equipment. 

9. - Maintains orderly, accessible records file (patient and HC), 

10. - Kepps beds open-j-if possible-- for emergencies. 

11. - Has private room for consultations with patients. 



4^. 
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E> Clinical Evaluation, Treatment, and Disposij^^^i^N 

(Patient Management) 

1. - Follows good procedure in admitting patients to the Health Center 

hospital and E.R., or in handling patients in Outpatient Department, 
(a) gets all needed specific data on presenting complaint 
and initial condition from .patient or ^n^rmant 
identifies the clinical problem quickly (clinical work -up) 
(c) decides (appropriately) to admit and diagnose rather * 
than refer 

determines what current medications patient may be on 
identifies patient risks, especially -crucial ones 
studies any records or notes, if patient has been referred 

to local hospital 
examines immediately in emergency and acute cases 
treats immediately' in emergency and acute cases 
records intake- data', initial decisions-, time, risks. 



(b) 



(d) 
(e) 
(f) 



(g) 
.(h)' 



2. - Institutes adequate plan of care 



(a 
(b 
(c 
(d 
(e 
(f 
(g 
(h 
(i 



(a 
(b 
(c 
(d 
(e 
(f 
(g 
(h 
(i 
(j 



history, personal and family, beyond just the chief complaint 
PE, general and specif 
laboratory, routine and special 

>studies records of previous hospitalization or H.C. treatment 
consults as necessary 

X diagnoses 'in view* of. the above (clarifies the problem) 
keeps complete records of all the above 
is a,ware. of risks In proposed treatment 
refers as necessary (See B 3) 



3. - Provides proper hospital treatment and follow-up. 



diet ^ : 

m*edication ' . 
recognizes complications and makes' the necessary decisions 
continued diagnosis, if ipdicate^ (re-evaluation of symptoms) 
checks non-responding patients 
monitors vital signs 

insures that staff know how to dispense medicines. Use -equipment 

insures proper supervision o.f pal^ireftt.is__care 

protects patient from potential avoidable complications 

'early mobilization ' 



4. - Provid|es proper OPD treatment and follow-up 

- checks agreement of diagnosis, lab work, and Rx 

- administers ^propriate medication 

- recognizes potential complicatiofis 

- does minor surgery under sanitary conditions 

- fol'lows-up depending upon Rx, severity, and nature of illness 

- insures that staff know how to dispense medicine, use equipment 



Disposes of the case on good grounds and with good advice 

(a) refers to base hospital (See B 3) or other agency (local) 

(b) makes appointment to a clinic 

(c) discharges after examination and study of records 

(d) prescribes medicines, diet, home care, exercise, hygiene 
(ej records disposition and reasons 

(f) makes appointment for cl^^ckup or repeated treatment 

(g) tells patient what side-ref fects to expect, how to avoid 
.> ' relapses, whether or not to return to work, etc. 

(h) advises patient on rehabilitation 

20 



y. Approacb and Attitudes. Toward Patients 

, 1, - Maintains good personal relationships with patients t (develops rappdrfD 

- knows names, addresses, .family conditions, jobs , problems 
* - talks W understandable language 

- visits patients at.hoiite (occasionally) 

- gives impress-ion of being a "family physician" 

2,<a)Tak^s family 's. economic situation into accQ,unt in relation to the 
costs 'of ' , . ^ . ^ 

- hospitalization \ ' ' ' 

* - drugs ' . , ' ' - . 

- laboratory procedures 

(b)Is aware^ of be'nefits ' available' to patients from various health-plans 
and other sources:- . ^* . / * 

- medicare 

- medicaid • ^ 

- social security, . . I * . 
• . . -^vocational rehabilitation 

- V, A/ ' ^ /' ' 

- private insurance « ."^ ^ . 

' fclEducates.individuals and' families re these benefits. , . 

X. 3, - Visits patients, as feasible, a^ expression of concern 

- at base-hospital after referral • - 

- at Health Center or *home' after treatment 



G, Productivity ' 

1. - Has minimal absenteeism 'frpm assignment 

2. -Maintains "reasonable" patient load 

?, - Keeps adequate and clear records «for all patients 

\ ' ^ ' - ^ 

4, - Meets all his scheduled clin-ic assignments 
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Participation and Communic a tidn in th e Local Community 

1. - ti^erts leadership »at the local level: . * 

- campaigns for preventrye clftiic5 (See A 4) , ' 
. ^ - takes initiativ^e rQ local problems (See A 2) 

- works effective^ly with community groups in discussing 

and solving local healtV problems- ^ 
. ' - involves non-medical persons. in discussing and solving 

problems 

- urges populace tOs^ take, ^vantage of clinics and education 



2. 
3. 
4. 



Gives lectures, demonst rat irons to, community groups 



Belongs tc5 local medical societies 



5. 
6. 



- Refers patients t6 other institutions in the community (family- 
counseling^, welfare^ schools, cflinic^ psychiatrist, the church): 

- refers to the correct agency 

- gives adequate reasons for referral 

- us^es proper channels ' ^ ' 

- keeps records of such referrals ? 

- Maintains open communications (Channels between community and H.C. 

»' «> * *' **' ' ^ ' 

- Partivcipates iri^all local groups that Vork for ear$y detection 
' of the major non-communicable .diseases and alcoholisni 



J. legislation and Department Requirements 

1. - Complies with existing l^aws and regulations: 
^ (a) reports communicable dieses- and'cancer - 

(b) checks on licensing of medical and parame'^dical personnel 
, . (c) delegates respohsibility. to those legal^ly qualified 

(d) reports deaths and births ^ 

(e) maintains all loca^. records and forms* 

(f) ^ prescribes and dispenses^drugs and narcotics* within the 

(g) issues health, birth, marria^ certificates after 

appitmriate PE ^ , ' 

(h) practices^ abortion^^within, '^tbe Law 

•''(i) uses appropriate channels for referrals • 
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Section II: Related Dimensions 



J. Physician Characteristics 

1. - Exerts leadership at local level (see H.l) 

2. - Is -satisfied in his work 

- professional 

- financial 

- peer and colleague relationships 

•3. - Is profiessionally honest 

- recognizes mistakes and limitations > . ' 

- is ethical in all medical matters 

- deals honestly 'with cornplaints , . * 

- deals fairly with colleagues * 

4.(a)Keeps informed of the latest developments in the profession 

- attends professional meetings 

- keeps up with the literature 

- reviews department and other statistics and reports 
(b)Makes self available "for self -improvement in knowledge and skills 

5/ - RelateS'Well to the community as a whole: 

- maintains commurii cation between community and Health Centei 

- cooperates with civic authorities 

- is seen as a positive fprce for better health conditions ^ 

^ - works with local private physicians 

- knows the community, its problems ,'%fttl its resources 

k. Patient* Satisfaction ^ 



1- - Feels that hospital care is adequate: ♦ 

- personnel 

- facilities 

- attention to needs 

* i 

2. - Feeis that costs of medical care at Health Center are reasonable 

3. - Feels satisfied that he has received g^od medical attention from- 

the physician ^ 

- thorough diagnosis 

' - physical examination (beyond loqus of original complaint) 

- laboratory work 

- thorougTi treatment 

- assistance in preventive practices 

- follow-up 

- referral to othe^ej^agencies 

4. - Is satisfied with degree of communication with the physician 

- has chance to explain his complaints - 

- is encouraged to reveal all pertinent information 

- is told the diagnosis 

- is told why lab work ii ordered , 

- is told reasons for and cautions with^ various therapies' 

- is- told about prognosis and rehabilitation 
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Section III 



Environmental factors affecting quality of judgment and performance : 
the working conditions and constraints constituting the physicians* milieu 
(Appears, to assume that the physician is a high-quality practitioner, and 
these conditions either enhance or hinder his performance). 



• ■ ) 

or local 



l/- 'Policy and priorities established by Department of Health and/or 
H.C. director (r/^lating to communications, admissions, personnel, 
referrals, treatment, finances, reports, clinics, morale, education, 
efficiency, regulations, etc.) *^ 

, ; 2.'- Ratios of" MP's, nurses, paramedics to local population (both^in H.C. 
and in local private situations) and to size of H.C. hospital 

3. - Availability^;i^^manent or on call or scheduled) of trained ancillary 

medical peopljj ^.-? f 
consultar^s and specialists 
sanitarian 
, preventive clinic directors 

medical records clerk or librarian 
health educator - ^ 
pharmacist or aide 
dentist k 

ambulance/ driver / 
social worker » 
me^Jical technicians 
school nurse 

4. - Patient load 

S» .T^ Referral feedback from base hospital or private agencies 

6. - Adequate,' sanitary, and well-maintained working space (consultation, 
nU'3Pse;ry, w^g^jt-ing, ejcaniination, operation, storage, lab) ^ 

1\ - Modern diagnostic and laboratory equip ment and facilities 

^sj. \ • 

^ ^8. - $i4>plies^ -and^:ispeed with which they can' be replenished 
9. - Transportation and communication available 

10. - Incidence of disease in locality, particularly serious diseases or 
^ those highly contagious 

\ 11. -- Hospital facility is adequate to needs and size of local population 

^ i / • ^ ■ 

12. - Number and type of responsibilities and assignments given to the 
. individual physician / 

r ^ . ^ . 

13/,. - Speed with wjiich service orders (lab, records-, pharmacy) are 

/ executed in. H.C. 

I 

14, - Geographical; location of locality, and of the H.C, and of- the 
base hospital] • ' . 

id S 
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Section IV ' 

Record Forms for Evaluation of PHysician • 
Handling of Presenting Complaints . 

V 

General Outline for Presenting Complaints 



Not.^ Not 
Essential Essential Done Done 



1. 
2. 
3. 
4. 
5. 
6. 
7. 
8. 
9. 
10. 

ir. 

12. 
13. 
14. 
1^. 
16. 
17. 



Admitting Complaint 
Admitting CompLaintv Explored 
History, Personal 
History, Family . 
Physical Exam, General 
Physical ^xam. Special 
Laboratory Procedures, Routihe' 
Laboratory Procedures, Special 
Consultation 
Diagnosis , Initial 
Diagnosis, Corrected - 
Diagnosis, Final 
.Transfer. 
Treatment 
Follow Up ^ 
Pi'otection of Patient 
Disposition 
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J Preliminary Form ^ 

CRITERIA FOR EVALUATION OF QUALITY OF ' 
MEDICAL CARE-OF.-THE PRIMARY PHYSICIAN IN . . . 

Pregnancy, tjeonatal Problems , 
Is medical record available? Yes No 

Maternal Factors Recorded (f^istory) 



Yes No 



1. Age and parity 

2. Duration of pregnancy - ^ 

3. Outcome/of previous pregnancies as 
to abortion/ intrauterine,' death , 
weight, and 'congenita)r anomalies 

4. Matctmal diseases * (diabetes, anemia, 
malnutrition) (previous^blood trans- 
fusion) 

5. Pregnancy period 

Prenatal care , (Place) 

Diseases during pregnancy 

Drugs used 



(Date) (ResUlts) 



Serdlogy 



Type and RH_^ 



(Date) (Results) 
^ ■ ■ 



6. Duration of labor 

1st stage \ hours 

2nd stage . - hours 

• * 

7 .'^ .BuptufS ^o^ membranes 

:(in relation to onset of labor} 

8. Prefsentation ^ * 



9. Method of delivery 



10. Date and hour of bi;*th^ 

' 1 
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Pregnancy and ^Neonatal -2* 



Physical findings recorded * 

Yes N(/ • . \ ■ ^ 

j ^ Apgar score 1 and 5 minutes • 

f • * Weight 

_2 ^ ' ' Sex . " ^ 

Complications recorded re high risk babies 

Yes No .Date § Hour Date Decision S Action Taken 
*_ • Identified Referred Correct Incorrect ' ' 

, , subnormal 

^ ^ [ weight (less 

than 5 1/2 lbs.) 

^ - Macrosomia (7 to 

' . -9^3 lbs.) • 

i , Jaundice 1st 

V ^ ^ \ 24 hours 

( , ^ ^ ' ^ * ' . > R espiratory diffi- 

' culty and/or 
cy^osis 

' • ' ' C onvulsions 

Infections 



_Persistent re- 
fusal of feeding 
and vomiting 

Abdominal dis- 
""•ten.tion 

_Hypo-hypert}^ermia 
(persistent) . 

Bleeding 



^3 



Congenital defects 
Cardiovascul'ai:' 



Hydrocephalus^ 



^ - -S pina Bifida 



Other <:^ecif yj^ 



Yes No 



ERJC 



CARE OF THE NORMAL NEWBORN BABY 
Feeding adequate (for breast feeding and/or formula) 
Weight recording is adequate 
Hygienic techniques adeqijalje 

^ DISPOSITIONS ON DISCHARGE 
Adequate instructions for feeding given 
Appointment to the Well Baby clinic given 
Adequate '^instructions for hygienic measures given \ 
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DEHYDRATION AND HYDftATING PROCEDURE 



Yes 


No 




Yes 


No. 




Yes"" 


No 





Age recorded 
Initial weight 



Recorded Correct Incorrect* 



1. degree of dehydration * ' 

'.\ • • 

2. etiologic factors. dejte,cted 

' - • ' ' ^ * 
3.. total amount of fluid in -24 hours 



4. electrolyte concentration and content 

5. route and raye ,of admi^si'ration 

6. responses ta thferapy 

7. final disposal ' " ' • • 
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^, t. TRAUMA, SHOCK, BLEEDING 

• I (1) Identification of the prpblfem 

'\ V ^ Av History- -from patient or informant 



B. Immediate evaluation of physical injuries* and severity in an. 
; ' orderly manner. (CheeJhist needed for type and site of injury-^ 

recognition, and disposition, ) JAlso indicate time elapsed ' 

since trauma occurred.) ' , ^ ^ 

1. Head and neck /' 

i 2 . Chest ' 

' 3. Abdomen ' ' . 

4. Extremities, 



Time of injury 

Time of arrival at H.C. 



TYPE AND SITE OF INJURY 



Head and Neck :■/' " vi^FT 

1. Scalp lacerations -| 

a. . With bone involvement 

2. Major lacerations with eJ^e ayid ' 
ear involvement 

3. Cerebral concussioji with brief, 
lo^s of consciousness 

.a. No'Tleuro fi^ndings 

b. Return of unconsciousness 

c. Focal neurological signs 
/ d. CSF leak (ear,, nasal) 

4. Fractures. 

' Linear skull -fractures --no 
s heurojogical ^igns. 

^b.- Maxillofacial 'without dis- 
placement, 

c. With displacements \ 

5. Penetrating irtguries of the 
neck . ^ 



Recognized 



Admitted 

and 
Rx at HC 



Referred 
♦ to 
B H 



• Time 
Treatment 
: Started ' 
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£• Trauma, Shock, Bleeding-2 I 



vs Admitted Referred Time 
TYPE AND SITE OF INJURY Recognized and to Treatment' 
: ' Rx at HC D H Started - 

* r - I 

Chest ' • 



9 



1. Contusion of the chest wall with 
simple^ rib fracture 

2. Multiple ri^ fractures (with 
flail chest) 

3. Prieumothrax 

4 . Hemothorax 

5. Hemopericardium 
Abdomen 

1. Blunt trauma with ruptured viscus ^ 
(stomachr- intestine, bladder) 

2. ' Concealed hemorrhage 

3. Penetrating injuries_^ 
Extremities ^ 

1. Joint, sprains 

2. Fractures ' , 

a. ^Undispla^ced (of pelvis and 

extremities) 

b. Closed, displaced i^"^ 

[ c. Closed with potential 1 
n^ui-.o logical or valvular in- A 
vol>^pment . - V: ^^^^^y^'^\. 



• d. Ofjen'' « ' ^ 

* 

3. Lacerations 

* ( ' a. Simple ' ^ 

j b. With 'tendon, nerve, or vascular 
* involvement 
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C. Traiiffla., Shock, Bleeding-3 

(2) Initiation of emergency management according to patient status^ at 
health center level. Use qhart below, and also indicate time. J 

TYPE OF THERAPY IN IMMEDIATE MANAGEMENT , ' . * 



Ess en- Cor- In cor r 

tial rectAect 



1. Initial evaluation* of magnitude of injuries 

2. Triage classification (minimal, immediate) etc. 

3. Therapeutic approach 

4. Establishment of airway 

5. Establishment of adequate IV route 

6. Type and rate of fluid administration 

7. ' Attempt to stop external hemorrhage > 

8. Monitor^ing of Rx approach 

9. Indicated laboratory te'sts 

10. Antibacterial Rx and tetanus prophylaxis 

11. Adequate wound cleaning and debridement 

12. Choice of initial or delayed primary closure 

13. Immobilization and bandaging of injuries 

14. " Type and timing of pain relief 
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. D. ABDOMINAL PAIN ^ f^^ 

I. Identification of problem: 

Oriented towardcthe recognition of an acute surgical abdomen, 
intestinal obstruction, GI bleeding or progressive intra-abdominal 
problem who might require surgical intervention within a short period 
of time. Differen.tiation of this patient from one that might require 
medical observatiQn or admission to the Health Center. 

A - History ; ^ 

fj' 1. Pt's age 

2. Onset and duration- - 

3. Location and radiation 

4. Character^- and intensity of pain ^ * * ' 

5. Previous history of similar symptoms and management 

6. Current medications 

7. Associated symptoms 

a) Nausea and vomiting 
^ " * hy Last B.M> ,^ v 

c). Urinary complaints 
^ d) L.M.P. in the female 

' 8. 'Previous abdominal operations? 
, 9. History of hernias 

10. Weight loss / • ^ • ' 

11. Change of nutritional,, bowel and bladder habits j 

12. Smoking and drinking hab'fts 

B - Physical Exam ; ^ 

t 1. B.P., P. , T and R # 

2. Jaundice 

3. * Palpable mass or organs 

4. Abdominal tenderness, guarding, rigidity, rebound tender- 
ness, abnormal peristaltic sounds, distension 

• 5. Femoral pulses* or inguinal masses 

6. Rectal tenderness or palpable masses 

7. Description of stdols 

'8. State of hydration i 
9.. Back and:^j;%Y; A. tenderness 
10'^ Scars • 

II. Plan of pare for the presenting problem. (E.g., determined that the care 
..is not surgical but deserves hospitalization in the Health Center.) 

-A - Initial Management 

1 . NPO . ' . * ' ^ 

. , ' 2. IV fluids ' ^ ' . • 'i 

'3. Nasogastric suction, if persistant nausea and vomiting 
" . 4* Monitoring of vital signs 
. \\ , 5 . . Considerations of relief of pain if no respiratory or 

circlilatory emtiar.rassment* 
6, .'^tibacterialSj if indicated 
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D. Abdominal Pain-2 
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B -Lab procedures 

'1. CBC . • ^ 

f 2. Urinalysis - ' 

^ 3. Flat plate and upright of abciomen ^ 

4. Chest X-ray \ / (i^ facilities available) 

III. Follow Up' 

4' Vital signs, monitor 

2. I and 0 evaluation of adequacy of hydration 
Z. Re-evaluation of syinpt:oms 

4. Removal of nasagastric suction when peristalsis present and 
disappearance of abdominal pain, .nauspa and vomiting 

5. Starting and progressive increase in feedings 

6. Recognition o'f. complications 

/ # 

a) Persistent fever 

b) Pain 

c) Persistent nausea and vomiting 

d) Development of rigid abdomen^*^' 

e) Appearance of jaundice oz^vfly palpable abdominal masses 

f) Bleeding 1 . 

g) Appearance of jabdomin^ distention 

IV. Criteria for Discharge 

The recidivism or benignity of process will become* self-evident 
within first 24-40 hours. Progressive development or relentlessness 
of symptoms make referral to base hospital mandatory. 
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Points to Be Considered in Overall Criteria 



liu. ♦ Identification of problem : 

A - Problems to be considered: 

a) Identification of airway obstruction' 

b) Evidence of secretions 
(5) bxigenation problem 

\d) Bronchial sepsis 

B Criteria for Admission to Health Centers ; I 

This depends on the facilities of manpower and equips 
ment after the identification of problems has been re- 
corded ahd orienVed. 

' C - Referrai Note 

e 

2. Adequate plan of care 

A - Problems- V 



/ 



1. Recognition of airways obstruction 

2. Secretion * . 

3. Oxigenatiop 
4^ Sepsis 

B.' - History 

1. Present and past history 
) * ' " 2% Establish criteria, for a<:uteness or chronicity 
\3. Pegree.qf incapacitation 

Chronic ' - ^ ; : 

^ - Acute ^ . • , - 

C - pliy-sical ' ' ^ \ ^ 

1. Expression of evidence^' 

' a) Toxicity 

, , ^ ' b) Physical , 

^ 2. Evidei^ce of , obstmietigp' ' 

V , 'a) Evidence o£ .parenchymal involvement 

! ^ ' by Evidence of arterial desaturation 

D Initial plan > j . , ^ ^ ^ 



1. Basic laboratory requirements 

. ' • WBC - 

, ^Hematocrit; • 
Cuitui:^ and stains, , . 
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E. Respiratory Complaint -2 ^ " 

2. .X-ray^ chest • 

(depending on facilij:ies available) 

3. Hydration as indicated * 
ft 

4. Antibiotic as indicated ^ - 

-Type 

-Criteria for initiating treatment 
, -Record time of*usage and discontinuation 



)f clinical findi] 



3.. . Adequat^e treatment 

1. Improvements ^of cl^iqal findings 

2. Clearing of physical findings * 

3. Clearing of Rx findings > 

4. Laboratory evidence of improvement 

4. Criteria for consultation - , 

* ' I'f improvement is not evident or if any of the presenting 

signs are worsened. 

5 . Criteria for discharge 

6. Final disposition 

Establishing^he presence of underlying chronic disorcjers 
that might have contribtited to the present illness 

^ . \ 

Appointments for follow-up clinical 

Follow-- up of historical, physical, X-ray and laboratory findings 



The following checklist of indicators for the evaluation of respira- 
tory disorders at the lev^l of primary physician was also prepared. 
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/ 

HISTORY RECORDED 



E. Respiratory Complaint - 3 



Yes 



No 



PHYSICAL EXA MINATION FINDINGS RECORDED 
Yes - No 



Patient's age 
Time and age of onset 
Characte^of onset' ^^-^ 
History of chronic cough if fti^lsst 
Previous episodes or first epEode 
Previous hospital care ^ 
Treatment used in the past or at present 
Radiographic studies done in the past ' 
History of- contact with ill persons 
History of specie procedures ahd 
treatment or -surg6ry done 



yiTAL.SIGNS .'Pulse for tachycardia' 
-BP 

, - -Respirations for tachypnea 
' * -Temperature 

DEFORMITIES OF 'chest - Bones and spine 

« retractions 

PRESENCE OF STRIDOR- . ' 

CYANOSIS ' , ■. 

HOARSENESS 

AUSCULTATION 

PERCUSSION 

^"JnJr QUANTITY OF SECRETIONS WHERE 
PRESENT 




^ .Uncontrolled fever 
~ Hemolysis 

Severe dyspnea and 

orthopnea 
I&ieumothorax 
Lung Abscess 
^ Congestive hea^rt 
failure 
ShocJC^^ ' " 
• Dehydration 
♦Allergic reaction 
Confusion and deepening 
cyanosis 
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E. Respiratory Complaint 



PLAN OF CARE RECORDED 

Yes No 



DISPOSITION ON DISCHARGE RECORDED 



1. Minimal laboratory 

A. WBC, Hematocrit 

B. Examination of sputum 

C. Smear and gram stain of sputum^ 
•D. Culture 

2. X-ray chefit 

3. Suction as indicated 

4. Hydration as indicated 

Type 
Route 

. 5. Antibiotics as indicated 
-Type given 

-Given as therapeutical i 
-Trial or after a stain 

was performeH 
-Length of administration 
-Time change 
-Why? 

6; Specific therapeutical 

-Measures 
-Bronchodilators 
-Steroid (criteria for 
giving it) 

7. Proper use of consultation 

-When nieeded 

8. - Referral when indicated 



'Yes No 



-Type of medication upon discharge 
-Diet given on discharge 
-Appointment given for H.C. clinic 
-Disposition considered adequate or in- 
adequate for condition 
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Fs Unconscious Patient^ and Convulaiona 

UNCONSCIOUS PATIENr 



Hospitalized 

At local Medical Center 
; Referred to UDII J 

Referred tp Private HoGpital 

If Hospitalized at Local Health Center 



Yes 



No 



U 
2. 
3. 
5. 
7. 

10. 



Associated with trauma by history 
physical' and/ojr work up 
Associated vi^ti bleeding and/or 
shock 

Serious respiratory obstruction 
Respiratory paralysis 
Urinary tract obstruction 
Evidence of meningeal irritation 
Associated with serious systems 
complications of any sort 
Evidence of fo^al neurological find- 
ings unrelated to strolce 
Ihe patient, recovered consciousness 
in 24 hours after admission 
The patient has transferred to UDII« 
within 24 hours 



( ) 



( ) 

( ) 

C ) 

( ) 



( 
( 



( ) 



( ) 
( ) 
( ) 
( ) 



Criteria for adequate initial evaluation: 
History 



History of as'soclated trauma 
History of drug ingestion 
History of ^exposure to noxious agents 
PrevJLous history o^f unconsciousness 
Length of unconsciousness recorded 
Family history of epilepsy recorded 

Criteria f or PE ' ' 

1« Vital functions recordec} 

2« Inspection and palpation of the body 

3« Size and reactivity of pupils 

Eyegrounds examination 

5« Signs of meningeal irritation 

6« Focal paralysis 

7« State of hydration 

Criteria for Lab^ and X-ray work 

CBC done- , , v , 

Urinalysis done 
X-rays of chest 
c X-rays of slculX . - 



( 
-( 
( 
( 
( 
( 



( 
( 
( 
< 
( 
( 
( 



( ) 

( ) 

( ) 

( ) 



( ) 
( ) 

S I 

( ) 
( ) 



( 

( ) 
( ) 
( ) 
( ) 

\ ] 



( ) 
( ) 
( ) 
( ) 
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^ ■ F. Unconscious and Convulisions - 2 

Criteria for adequate initial plan of care 



1. 
2. 
3. 
4. 
5. 
6. 



Orders for clearing airway 
Vital signs monitoring , 
Feeding orders- 
Intake and output 
Consultations 
Excretory function 



Criteria for adequate follow-up care: 

1. Vital signs aonitorins .* 

' 2. Ctatus of cardiopulmonary state 

3, Status of excretions 

A. Early mobilization 

St Adequate care of complications 

6. Status of hydration 

Criteria for discharge and disposition 

1, Adequate transfer note to UDH 

2, Patient feeling conscious and .ambulatory 

3, Febrile 

f;. Feeding problems 

5. Ho systemic complications 



Yes 

( ) 

( ) 

( > ) 

( ) 

( ) 

( ) 



( ) 

( ) 

( ) 

( ) 

( ) 

( ) 



( ) 

( ) 

( ^ 

( ) 

( ) 



No 

( ) 

( . ) 

( ) 

( ) 

( ) 

(. ) 



( ) 

( ) 

( ) 

( ) 

( ) 

( ) 



( ) 

( ) 

( ) 

( ) 

( ) 



Criteria for admissions: 
Acute Convuls/lons 
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!• First convulsion not associated with 
increased intracranial pressure, nuchal 
rigidity or subarachnoid hemorrhage 

2. First convulsion not associated to 
trauma prolonged unconsciousness or focal 
findings 

3. First convulsion not associated with 
^ serious organ or oysteoic disease 

'chronic Convulsions 

1. Not responding repeated seizures 

2. Already work up at the Regional Hospital* 
3* Anticonv^ilsants are available 

Associated with »tatufl^^)?ilepticus^ 
5. Relatfed to head injuries ' 
' • * 

Criteria for adequate pl^Ih of care: 
History: " ^ 

Jldstory of trauma or drug inc^stlon 
Focal onset 
Generalized seisure 



( ) 



( 
( 
( 
( 



( ) 



r 



( ) 

( ) 
( ) 



( ) 

( ) 

( ) 

( ) 



( ) 
( ) 
(. ) 
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P. Unconscious and Convulsions - 3 



Associated vrlth exanthema 

Associated vlth severe headache or signs 

of Increased Intracranial pressure 



Physical Examination; 

1. ^Itai Pigns » 

2. Condition of hydration " 

3. Condition of skin 

4. , Signs of meningeal Irritation 

5. Size and response of" pupils 
(r. Focal paralysis 

• 7. Eyegrounds examined 
C. Focal paralysis 



Yes 
( ) 
( ) 



( ) 

'( ) 

) ) 

( ) 

^ ) 

( ) 

( ) 



Adequate Care; 



i 



!• Orders for clearing airway 

2# Vital signs mopitoring 

3# Protection against body injuries' 
P2 therapy is indicated by cyanosis 

5. Glucose IV 

6. Mobilization early 

7« Effective anticonvulsant drugs 

C« Reports for abnormalities of level 

' of consciousness 



( ) 
( ) 
( ) 
( ) 

( ^) 



I Criteria of Lab, is the same as for unconsciOfua 'patienC' 



COMPLICATIONS IDENTIFIED 



No . 
) - 

y 



( ) 
( ) 
( ) 
( ) 
( ) 
( ) 
( ) 
( ) 



( ) 

(. ) 

( ) 

( ) 

c ) 

( ) 

( > 

( ). 



Action Tnk e 




Dgte Recorded 



Correct 



Incorrect 



1. 

2. 
3. 
6. 
5. 
6. 
7. 

r» 

9. 
10. 
11. 
/12. 



Depressing of IfVfel of 
consciousness 
Status epilepticu 
UrirJfery obstructldj 
Respiratory distress 
Resp^atory obstruent ion 
Dehydration 
Pneuciionia 

Abdocninal distensio 
Urinary tract ihfecH\Lon 
Status of the skin 
Decubitus ulcers 
Signs of phlebitis 



) 


( 


y 


) 


( 


) 




c 

' ' ( 


) 
) 


, ( 


) 


) 


( 




) - 


- ( 




), 


( 


> 


) 


( 


) 


) 


( 


) 


) 


( 




) 


i 
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£• Unconscious and Comrulsiona • % 
POLLWJ-UP CARE AND^ DISCHARGE 



IGE (this application for both, the-cnconscious and the 
, , convulsinc Pare) ^ ^ * 



Follow-up Care:, ' Yes ' No 

1. /i,ionitoring of vital'^signs . ( ) ( ) j 

2. Mobflization of the pat5^ent ( ) ( ) • 

3. Reported observationa of level ^\ ^ 

of consciousness ' '( ) 

4. Laboratory and X-rays reported 

and, reviewed ' ( ) ( ) 

5. Level of hydration ( ) ( ) 

6. . Referred to medical center 

in time and adequate ( O ( ) 

Criteria for Discharge; 



1. No fever or systemic complication ( ) ^ ^ ( ) 

2. No signs of meningeal irritation 

recorded ^ ' ( ) ' * ( ) 

3. Feeling conscious and no mental " 

disturbiince ( ) ^ ^ j^*^ 

A. Anticonvulsant prescribed for follow > 

up in the case of convulsions ( ) ( ) 



only good evaluator is a physician who studies the record--and the criteria 
of quality practice in effect are in his head. The majority view was that ' 
it is possible to determine adequate criteri'a in advance and that *'evaluators" 
may be merely data-gatherers who record according to predetermined check- 
lists, regardless of the mode employed. The record form developed for *'preg-- 
nancy and neonatal problems" provides an example. On the one hand, much of 
what is to be checked is a matter of redords and an indication of their com- 
pleteness. But in the case of complications, there is the added need for a ^ 
judgement to be rendered concerning the correctness of either (a) the local 
action taken or (b) the referral made. (See page 21). 

Another observation concerns the terminology used. The words "appropri- 
ate," "necessary," and "feasible" appear repeatedly in the criteria, and of 
course are qualifiers which have not yet been defined^Kb precise performance 



terms. In many instances the intended "valued performance" would depend upon 
circumstances, and it may be that in any case a certain degree of subjective 
assessment of quality would have to obtain. (See the point of view referred 

to in the paragraph just -above*) • ^ 

I. 

; In other criteria, the verb "to be aware of" is applied. To ,be knowl-' 
edgeafrle and aware is necessary to proper action, but is not usually suffi- 
cient in itself to insure of /^verify that needed action is taken. In a few 
instances, the "being awa're is the apparent intent of the criterion, and 
is sufficient .for what the Conference intended. ' ' 

A related cdncem is tjiat^^ven whe<e more action-oriented verbs are em- 
ployed, the level -of acceptd^la performance is notr-indeed, ia some cases, 
may not be--indicated. In the c^riterion/ "Conducts regular immunization 
programs," there is not yet ai^tand^rd for regularity, Similarly, as relates 
to physician visitation, it i-s apCx^ear w[het1ier one visist to a patient--at 
home, in hospital, at work--wouH^^^ati*sfy 'ihi?;^ criterion, or whether somehow 
a sefies^of visits'^is intended. WCitftfe^^other. hjihd dimension cbA- 

cerning productivity, one criterionJ^lVstated'^asV "Meets all' scheduled* 
clinic assignments" and that is an ab^^ute and presumably measurable requirer 
ment, .easy to determine, and to record. > V V / 

• ' ^ ' • \ \ 

One observation made by Conference i^^^g^ reflected in the 



criterion outcomes, was that in some instances it would be necessary to, have 
less than absolute expectations. Where the physician is asked to "insure 

* 4 . 

I " ^ 

that all who should attend preventive clinics in fact do so," for example, 
the requirement is so stringent as*\o possibly defy adherence much less 
measurement. In cases where the "demand " is meant to be so precise, though, 
some crite/iofi behaviors (which reveal concern, attitude, or action) would 
need to be listed-'-even if only one of them was needed to satisfy the require 
jnent. 

SUBSEQUENT REVIEW and REVISION ' 

^ 'The compilation just discussed was submitted to the Core Committee and. 
Curso staff for review and revision, which occurred in^ several stages. 

The first step was to reorganize the outline so that two categories we^re 
removed from "Dimensions of Care". and placed under "Physician Qiaracteristics 
These were the sections concerned with Participation in the Community, and 
Productivity. It was difficult to determine just how to Classify these two 
dimensions, since both included traits directly related to actual, care (e.g., 
keeps adequate, clear records--and refers patients to other agencies...). 
Howev^, most behaviors outlined here were more related to the physician's 
characteristics and sense of responsibility. . » ' • 

At the same time, minor changes^were made in headings,- a few elements 
were placed elsewhere in .the taxondmy, and tWo specific indic^ors were re- 
moved, from the outline. \ " ' . 

In the* second stage, vai'ibus members of the Core and, Curso groups ^set up 
criteria by which to review the-cjOmpi*lation systematically., -They determined 



to: 



-rempve .all ppssibje duplication*^ ^ ^'^ 
-reorganize the;topical alignment^ ^ . • . 

-iletain those cfiteri^ which represente^l nvinimal rather th^ ' 

ideal expectations, to match -ipealistiiS situations^ ^^^^ I 
-remove items relating exclusively^ to the function of t5he> 

LHC director ' ' ^ 

-restructure the taj^pnomy so that it, would describe "the 

general physician anywhere",3'knd. not 'just the LHC physician 
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Working l&id^f these requirements) members of the Core Committee met on several 
occasions and concentrated on "the content and topics which they could pull 
from the earlier compilation; there was little emphasis at that time on re- 
finement of the .performance descriptors or possible levels of performance. 
To be sure, the Committee considered the^ specif icity of the statements along 
with measurement implications, but the main concern was appropriate "topical 
coverage.* ^ , / - 

Four- major areas wete identified in this restructuring: Community, Health 
Services,* personal Health Services, Personality (physician characteH'^tic^ , 
and Situational .Influences on Physician Performance. This resuitedv'lrf th'e 
elimination of four dimensions which had emerged yfrom the origihal conference, 
and a number of subgroupings of criteria. 

The third step involved reconsideration of all thje' preceding review/ 
revision work at a formal meeting which made use of input from other medical 
and measurement personnel. Two days were devoted to the detailed study of 
the changes already made; to debate,, rewording, new reorganization, and 
measurement implications; and to a determination of the priorities. In ^ 
particular, the original overall compilation (see pages 11-19)* was analyzed 
again and structured into final form in terms of Core Committee consensus. 

The final outcome is presented on the following five pages, and rep- ^ 
resents the formal report adopted by the groups involved. To be sure, there 
were some areas left incomplete or about which there was some uncertainty; ' 
and it was -recognized that work was *still needed on specificity, coverage, 
levels of expected competency, and the like. This final product automatically 
includes the record-forms earlier developed for several presenting complaints--"^ 
althpu^ that section .is not repeated here sirfce there was no opportuhity 
for the ad* hoc hospital group. to revise this material. The original--2UTid 
thus finals-record forms will be found on pages 20-36. ' ] - 
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\ CRITERIA FOR EVALUATING QUALITY OF MEDICAL CARE j 



COMMUNITY HEALTH SERVICES 



/ 



A. Community and Family Protection 

1. Keeps informed of environmental and social factors that affect , 
community healthi an.d of action that, could be taken_. 

a* coramunicable disease outbreak (e,^. , /hepatitis , influenza) , 

- traces source 

- immunizes 

- keeps needed medicines on hand 

- follows patients and contacts 
.b. nutrition needs 

c. overpopulation problems 

- counsels families 

, - conducts family planning program 

- refers to existing a^^encies /' 

d. social pathology problems 

- alcoholian 

- drug addiction 

- divorce 

- - adopt i^^ ' ' , 

- battered child 

-.children's neglect ^ ' < 

e. sanitary facilities and conditions ' 
f • aix and water pollution _ • 

g. major non-communicable diseases.. 

2. Takes jneasures to correct these factors where feasible; 
a. communicable disease outbreak (see criteria above) 

* b. nutrition needs « " ^ 

* c. overpopulation problems (see criteria above) 

.3. Conducts regular immunization programs for smallpox, DTP, poldo ^ 
and mea^les • ' v.. * 

"4. Participates in general educati9nal programs and efforts for 

individuals, for community, and for other heSlth care personnel 
(before and after problems erupt), concerning: 

- communicable diseases 

- sanitation* and pollution 

'^'^"^ ' - major non-co^unicable diseases 

* - family planning 

- nutrition • ^ 



5. Participates in programs aimed at eai^ly detection of heart 
disea.se, cancer, TB, glaucoma, . VD, diabetes 
a, actively participates' in clinics * 
. b. actively participates in educational efforts 
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I. COMMUNITY HEALTH SERVICES continued 

B. Legislation, Requirements of the Department of Health 
1. tomplies with existing laVs and regulations 

a. adequately reports deaths and births 

b. prescribes and dispenses drugs and narcotics, within the law 

c. reports^on TB, syphilis, hepatitis, gonorrhek, cancer 

d. reports on communicable diseases ' 



C. Participation arid .Communication within the Total Health System 

r^^. 1. Knows proper channels for referrals, whether ^in private ''system' 
or Department of Health systg 



2. Knows how referrals are handled at hospitals and other agencies 
3,. Makes appropriate use of referral- systems 

a. -refers at local l<evel where, ifeeded 

- from E.R. to OPD ' 

- tct local private hospital 

- to appropriate paramedicals 

b. refers to base (regional) hospital from E.R. or OPD 

- refers directly by, name 

- pre]^ares referral himself (not left to a paramedical) 

c. j)reparl^S; referrals which: -^^r- ' 

- ar^*|egible 

- contain* history and physical findings 

- include reasons for referral 

- include his diagnostic impressions 

- show that he attempted to diagnose 

- includ^any therapy already given 

d. takes patient risks into account 

e. asks for specific feijedback from hospital or other 'kgeitcy 

- arrival of patient 

- medical report 

- seen by right specialist - - ' • 

f. makes only necessary referrals 

g. refers to. appropriate specialist or office or institution 
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PERSONAL HEALTH SERVICES IN HOSPITAL AND OPD 

A. • Maintains adequate clinical records 

1. relates history and P£ to chief complaint 

2. diagnosis, lab results, and therapy agree in terms 

of chemical data 
•* 3.^ .disposition is made clear 
'4. referral notes are appropriate, complete 
S. follow-ups are done (depending on severity, therapy, 
diagnosed illness) 

B^. • Keeps clinical records available, organized, legible, complete 

£. Initial work-up. 

1 . age . ^ ' , . 

J 2. gets alj needed specific data on presenting complaint 

3. durat-ioiTof i-nriess 

4. associated signs and symptoms in adequate sequence 

5. deteimines previous and current treatment (e.g.^ medication) 
.6. other pertinent history 

7. records PE^ihcluding pertinent positive and negative findings 

8. identifies ^he problem(s) 

PTan of care • 

"../•• • > 

:,1. ini-t'ialf decisions recorded 
. ^2. 'treats Wiediately in emergency and acute cases' 

S. orders reT^ant laboratory work ^d procedures 
*^*.^ 4], consults as necessary 
* 5. provides* proper treatment and follow-up * 

a. diet 

b. medioeition 

c. recognizes and prevents potential medical complications 
anct^makes^ the necessary decisions (e.g., early mobilization) 

d. contihued diagnosis, if indicated (re-evaluation of symptoms) 
^. insures proper supervision of patient's care (e',g., minor 

surgery under aseptic technique) 
f* protects patient from potential security hazards- a 
' g. follow-up, depending upon treatment, severity, nature of 
problem 

E. Disposition of case 

l»..djL^s charge orders record^ed, with statement as to medicines, diet, 
' home care, exercise, hygienic measures 

2. orders appointment for follow-up clinic 

3. plans for rehabilitation and records it 
4".- advises patient regarding... 

• ^ a.. nature and duration of medication and treatment 

b. nature of illness 

c. side-effects of treatments 

d. how to avoid relapse - ^ 
;e. types of activity permissible 

f . methods of self-care ' 
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III. THE P^SICIAN as' A PERSON: PROFESSIONAL .ATTI'TUDES § CHARACTERISTICS 

A. Physician characte::istics . ^ 

1. Derives satisfaction fyom his work 

a. professional * . ,^ 

b. financial - - 

c. peer and colleague relationships ^ 

2. Keeps informed of latest^ developments in the profession 

a. attends professional meetings (enumerate).- - 

• b. keeps up with the* literature (subscriptions, etc.) 

c. reviews Department and other statistics and reports 

d. avails self^f formal ri&srf uctional. opportunities'for 
self-improvement in knowledge and skills 

3. Relates well to the community as a wRole 

'a. maintains communication betweerT^'ommunity and private 
practice and/.or Health Center ^ 

b. cooperates with civic authorities 

c. takes initiative- in working for improved health conditions 

d. work^ with all lo'cal physicians, in private or public practice 

e. knows the .community , its problems, and its resources. 

B^. Participation and communication in the 'local community 

1. Exerts leadership at the local level ^ 

2. Gives lectures and demonstrations to community groups 

3. Belongs to local medical societies 

C_. Patient satisfaction 

1. Feels satisfied that he h^s -received good medical attention 

from ^the physician , * * ^ 

a. physical examination (beyond locus of original complaint) 
^ b. tre«tent 



q. assistroce' in preventive practices ' 

d. follow-up (including hospital) 

e. referral to other agencies 

Z* Is satisfied with -degree of communication with the physician 

a. has chance to explain his cjomplaints i 

b. is encouraged to reveil all pertinent information 

c. is told the diagnosis 

d. is^ told why lab work is;ordered * ; 

e. is told re3<5ons for and cautions with various therapies 

f. is told about prognosis a|d rehabilitation 

3. Is satisfied that there' Is minimal waiting time and delay 
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SITUATIONAL INFLUENCES ON PHYSICIAN' PERFORMANCE^ ^ 

A. Policy and priorities 'and-^assignments established, by Department of 
Health and/or local H.C. director (relating to coinmunications , 
admissions, personnel, referrals, treatment, finances', reports, 
clinics, morale, education, efficiency, regulations , organization 
of resources) 

B^. Ratios of MP's, nurses , paramedics to local population (both in H.C. 
and in local private situations) and to size of H^.C. ^hospital 

C. Availability (permanent or on call or scheduled) ,of trained ancillaiy 
medical people ; 

1. consultants ^and specialists / - 

2. sanitarian 

3. public, health unit director ' ' . • ^ 

4. medi<:al records, clerk or librarian 

5. health educator 

6. pharmacist/br aide 

7. dentist 

8. ambulance drivef 

9. social worker ' >• . , * 

10. medical technicians \ • ' 

11. school nurse ^ , ^ 

D. Safe, sanjLtary,- available, and well-maintained resources 

1. Working space (iiitluding consultation, nursery, waiting, operation, 
' examination, storage, laboratory)' . 

2. Diagnostic and laboratory equipment (X-ray, EKG, suction, oocygen) 

3. .Medical, surgical', and other supplies 

4. Transportation for patients (emergency, referrals, and supplies) 

5. Hospital facilities and equipmentr, number of beds, ^tc. 

E. Patient load (S-IO patients per hour) 

£. Absenteeism (10% a/:ceptable; above tl]at, * totally unacceptable) 
G. Cbmnuinication facilities, both local and regional , 

Weekly schedule of physician, related to his responsibilities 
I^. Efficient bed utilization depending on admitting complaint 

1. efficiency in handling case 

2. service orders , ' ' 

3. consultation - • .* 

4. surgical facilities* -r 

5. equipment * . — —-1:^^*?:": 

J^. Geographi.cal location of LHC and travel time to base hospital ' 



COMMENTARY ' ^ . ^ . 

^ It ' 

The final guidelines and criteria^ are revised considerably from the 

initial conference output As noted, the Core Committee operated under a 

different set of assumptions whl^n undertaking the terminal revision, partic- 

ularly (a) moving from ideal expectations to more realistic ones and Cb) 

changing emphasis from the Health Center physician to the general physician i 

any setting. , - ^ ' ^ 

The revision resulted also- in a reduced number of dimensions as well 
as the 'elimination of^'certain criteria deemed either unessential or un- 
measurable. Moreover, the reorganisation has subsiuned certain dimensions 
into others, reflecting a point of view about practice and assessment as 
[well as the difficulty the Core group encountered in -dealing with dimensions 
as separate entities, ' ^ • . ^' ' 

A great measure of the final product is still of course topical in 
nature, and not behavioral. It is anticipated that .a good deal of further 
work would be neede'd-^as other groups 'engaged^ in siinilar endeavors have 
experienced- -to put the material into actual criterion form based on per- 
formance indicators. In a number of topics the criterion behaviors are 
not yet stated nor are J^^ls of performance^ indicated; whereas other topics 
have been amplified by^ statements of actions to be taken,,{See, for example, 
on page 40^ the criterion indicators 'for "takes measures to correct com- ^ 
municable disease outbreaV--and pagd 41 where requirements are outlinecj^ 
for the proper preparation of referrals^; ) ' ^ , ^ , ^ ^ 

• - Another observation iV that-? -for; tffe present anyway — even certain, 
topics have been rfembv^d from the.ta^^nomy which might later be reinstated. 
For example, the hapdlmj of prevjehtiye clinics. in the LHC; the propier use • 
of medicines; certain of the legislative or departmental requirements; the 
^distinctipn between OPD and hospital practice. As any further work may be * 
done with the critefiaj, it may be desirable to develop a section specifically 
for the '^special'* responsibilities of LHC physician. Actually, a nymber.pf 
such 'references exis't in the final Committee document, but are not yet organ- 
ized into a special section. \ ^ , . ^ . ■ 
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The present struct'ure may certainly serve as a base for additional work- 
in specifying performan'ce standards and thereby coming closer to a statement 
of what is', and is not, feasibly measurable- -and by what means. The speci- 
fication of particular behavioral indicators (with attention to sequ^ce, 
level of operation^ different but equally satisfactory criteria, and organi- 
zation into acceptable dimensions) is a difficult and continuing challenge. 
With respect to the several/'presenting complaints," of course, a^ good deal 
has already been accomplished in this direction, where, for example, the 
search for complications is emphasized (through listing specific topics or 
behaviors), and the unique elements of diagnosis arejyrt^ined (as opposed 
to* a ^§:Bneral work-up"). Different approaches are utilized for different 
complaints, however, and further study might result in consensus on format, 
sequence, mode of recording, and most importantly, the areas of judgement 
variously included in the present outlines (essfential, not essential; done 
correctly, done incorrectly). 

POTENTIALfUSES OF CRITERIA ^ 

^ Several values would appear to inhere in the products of this activity. 
Whether they are topical outlines, specific performance indicators, or dimen 
sions of the provision of medical care, the outcomes are potentially useful 
in variou^ways. (Actually, the activity itself may have had positive re- 
percussions, in that (a) a large group of medical personnel met and pro- 
duced a common product, (b) a first step was taken in the difficult task of 
performance specification, (c) attention was paid to the measurement impli- 
cations involved, and (d) a number of Puerto Rican medicos became better 
acquainted with the University's special Curso de Perfeccionamiento and its 
aims * ) 

If all the products are considered--from the initial compilation of the 
Conference work, to the final revised material, to the disease-oriented per- 
formance requirements--then the following potential uses may be listed for 
the topics and criteria: * * . 

- - -They provide a framework for developing an assessment approach 
to the work of physicians--in general practice or that group 
- spe,cifically located in the local Health CeilterSv 
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2. There is potential here--through the actual products as well 
as the process which led to them--for assessing the quality of 
care provided by other groups of physicians, as well as that pro- 
vided by the system (s) of which they may be a part. 

3. They may aid in emphasizing, in regular and in continuing medical 
education programs, the heajth-consciousness and patient-con- 
sciousness which are- increasingly important^at this time. 

^ * 

4. There is possible utility in -the development of, various course 
guides aftd outlines, and they may aid in determining priorities 
in continuing education programs. 

5. For the Department of Health, Jthere is a potential use in 
developing definitive job descriptions for those assigned ip 
LHC's or other facil-ities. * ^ 

6. The Department might also see a possibility of assigning 
physicians in terms of the dimensions of care they excel at; 
this of course is a long-range potential^ b^t even at this time., 
selective 'assignment according to preferences or indicated 
skills might be considered. 




7. Other potential ^applications in the Department are their use in 
various training ^programs , 'and in the standardization of record- ^ 
keeping procedures or referrals. 

8. The tropics and criteria may be useful, at a^ later time, in any 

iongitpdinal study of physician ^training and performance undertaken 
by any group on the Island. 

9. The products of course might be valuable to individual physicians 
to use for even a beginning at self-assessment in general practice. 

10. The Medical School, i« any wprk in new curricula, might -match 
the topics ana criteria against the objectives and cJfcen^Lpf 
the regular prograjn--whether for general preparation or speci 

. zation. This might highlight any needed revisions in the medical 
curriculum, or possibly in priorities or sequence. 

11. The continuing Curso de Perfeccionamiento (now known as the Curso ^- 

de Actualizatitfn Meliica) may find clues for restructuring the course 
or its emphases, or in "providing particular training in community- 
health areas. 

N . 52 

1-0 • 




APPEHDIX A; 



Special Committee for Development of Criteria 
of Competence in -Medical Practice 



Sub commit t 
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A. Hospital Medical Practice 

' 1. Carlos- Girod, M.D. (Chairman) 
* 2. Mr. PrSxedes Norat 

3. Margarita 'CSceres Costas, M.D^ 

4. Mario Garcfa Palmieri,_^M^4)r 

5. Luis Sanchez Longo, ?r.D. 

6. Juan J. HefnSndez Cibes, M.D. 

7. J*. A. 'Alvarez de Choudens, M.D. < 

8. Nydia de Jesds^ M.D. " 

9. AnibaJ Lugo, M,D. n 

10. ETS - Dr. Carl Bf -feyO^, M.*D. , Consultant ' 

11. PROJECT STAFF - Dr. Francisco" Veray 

B. Ambulatory Medical Practice j ^ ^ . 

1. Manuel Soto Viera, M.D. (Chai^rman) 

2. Jos^ lUanez Morales, M.D. 

3. Luis'S., Miranda, M.D., Chairman of the total Special Committee 
, 4. Lillian Haddock, M.D; 

5. Fr^cisco Oliveras, M.D. 

6. Marta'l. Valcarcel, M.D. 

7. Ralil Costas, M.D; 

8. • Ibrahim Perez, M.D. ' ' ' . 

9. ETS - William KastrinoJs., Ph.D., and Ennio Belen-Trujillo 
•TO. PROJECT STAFF - Egidio &^ Col5n-Rivera, M.D^. 

C. Community Medicine 



* 1. Rafael Rivera Castano, M.D. f (Chairman) 

2. Guillermo Arbona, M*.D. 

, 3. Jorge Fernandez, D. D. S. % 

4. Herber Rosa Silva,,WD. 

5. Antonio Ortiz, M.D. 

6. Juan A.- Rasell6, M.D. 

7. RamcJh Santini, M.D. 

8. JosS A. Rechany, M.D. 

9. ETS Roderick Ironside, Ph.D. ' 
10. PROJECT STAFF - Ariel Diaz, M.D. 



* Indicates members of the continuing,' or "core," committee. 
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THE CRtfTERION- PROBLEM IN THE 
EVALUffTIClN. OF PHYSICIANS , 

Ij. a. Davis 

/ 

The Medical School o^the University of Puerto Rico has forty-eight ; ' 

physicians with foreign medical trailing or foreign licenses who are just now - 

completing an ambitious six-month [fcurso de perfeccionamiento'' or retraining 

program. It is expected that theyiWill serve principally in Ijsland health 

centers where the need for physicians is acute. G(jbd^pedagogy--and the sponsor 

(the National Center for ^ealth Services Research and Development) --demand 

\ , • ^ ' 

careful evaluation of this new and creative solutiort to a passing manpower 

nee<i, for this is but the first of ai series of programs. 



The doctors in the Curso have t^ken the ilicensing exams of the Board of 
^Medical Examiners at least once, and all have failed to pass this minimum 
Standard. With the course behind them, 'they will shortly retake the licensing 
exams, and certainly their performance now^after retraining will^^^^KKe some 
evaluative criteria as to the general eff ectivejiess of the Curso. 

\ ' • 

Other evaluative possibilities were built into the prograpi from the 

beginning. As many'of you know--becaus^e you helped wijth the effort'--twelve 
tests in as many basic science or clinical science ^reas were developed by 
select faculty of the Medical School, following careful definition and speci- 
^ fication of what the critical knowledge and abilities in each- of these areas 
should be. Jhese tests w^re administered to the studerit physicians^ before 
the Curso began,. and an analysis of incorrect answers was provided^to the pro- 
gram planners and administrators. Now, with the first coursje ending, the 
student doctors will (this next Saturday) retake these tests, and we will , 
then be able to observe for these physicians, separately and collectively, 
the changes that inay have occurred, and also the areas >yhefe no change has 
taken place. ' - 7 

Although these two evaluative possibilities--p6rformance on, the licensing 
examinations and^on the. retest of the special UPR Medical Knowledge Tests-- 
seem quite valuable, there are still some^ rather sei:ious problems that we feel 
cannot (like El Pato) be ignored. 
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To cite. an example: There is^ a carefully suppressed study at an elite 
' Engineering school in the northeastern United States that found a negative 
or* inverse relationship between tesf-based grades in the program and later 
success on the job as measured by supervisors' ratings. Performing well in 
college, or on tests., may not be the ^ame as performing well in the'real life 
sit^uation. ' * ^ , 

Another aspept of this limitation is that tests are infested with some- 
V ^ thing we may call not too inaccurately' "t^st-taking ability/' A study by 

Dr. fedward Cureton of the University of Tennessee found that 90% of the 'reli- 
able variance in .tests of achievement (after specific training) could be 
explained, by differences in scholasUt aptitude unrelated to the training that 
had been- given. 

A more critical limitation -of leaning too heavily on the evaluation pro- 
cedures outlined thus far is their possible contribution to forces that 
pr^escribe ''teaching to the'^te^t" rather than to the complex activities and 

, responsibilities of the physician in practices-activities and responsibili- 
ties many of which are, probably impossible to transfer to paper-and-pencil 
Resting. This is the major flaw in one other program of retrain-ing physicians 
in the States, where the goal is training in &iglish languag^ facility and . 

. in ."test- taking/" 

For the purposes of the UPR program go beyond aiding some unfortunate 
'people to attain a value credential. .Principally, the goal is to improve the 
quality of medical care on the Island, by providing a. new supply of capable. 
' physicjgns in areas of critical manpower shortage , at a reasonable cost, and 
in a short time. 

This means that we are concerned not only with what the -physician knows^ 
.and can report on tests, but also with what the physician "does in his pro- 
fessional service. Our basic question becomes "What is a good" physician?" ♦ 
and this we must answer before developing measuring techniques to study and 
attest varying degrees of goodness, and to provide feedback for improyement 
of the Curso. ' ' ^ 
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• •» ' , ' 

I am generally distressed to find that if this basic question is posed 
(by myself to a physician of some well-earned reputability, or by those wh6 
have previously reported attempts to evaluate quality of nredical practice) 
some common and pimple problems emerge that assure the basic question will 
not be answered satisfactorily. One of these is the tendency to, think of 
**goodness" as a unitary whole, when in fact there are many different dimen- 
sions of goodness. Our question becomes (in term*s of the tolal activities 
of the physician in practice) "What are good qualities, or for that' matter, 
what are good qualities in particular situations?" physician may be good 
in one type of ^activity (e.g., diagnosis) and poor^in another (e.g., patient 
care); or, a particular quality may be relevant in one situation and not in 
another. The task before this distinguished gr6up is the specification of a 
variety of dimensions, that together bear on effective and efficient health 
care. . / * 

Another common difficulty, that I believe the literature review you have 
been provided will show, is that we tend to be vulnerable to taking those 
particular convictions of thipgs. we believe w£ do well personally (and that 
other physicians do not), and equating them to the most essential components 
of quality. As different individuals haVe attempted to specify or assess 
essential qualities, they have developed very different constellations of 
qualities (if indeed they do not flounder on the error of assuming unidimen- 
sionality!): 

It therefore seems critical that in the 'task before us we welcome >and 
relish differences among us in what begins to occur first in our individual 
thinking as important qualitative aspects. We are blessed here with many 
unique capabilities and talents; we are secure enough to express our convic- 
tions openly. But by the end of our three days, we need a varied schema for 
criterion specification, and a structure that the measurement technician may 
begin to help you transpose to reliable assessment procedures.* In short: we 
should not, must not, refer this problem to the specialist--either medical or ^ 
measurement specialist-rbut must assume collectively the responsibility, 
and trust that varied points of view and group commitment will produce a more, 
satisfactory answer than any heretofore achieved. 

There are several kinds of structures for qualitative dimensions that we 
may consider. One would be concerned with the content of practice. ^ This might 

r 
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have components of technical competence in given frequent situationSv, 'effective 
ness of treatment, appropriateness of referral, and the like. 

Another kind of structure would be that which focuses on the delivery of 
services. What are the numbers of patients treated, or what is the state of 
community health visibly related to the physician man-hours devoted to various 
activities? That is, "quality" may seem to demand looking into the eyegrounds 
of every presenting case, y6t rteduce the number of patients that can be seerf 
m a given period of time; or too good physician-patient relationship^^ may/ 
attract many patients with superficial complaints. 

In our consideration of structure for our criteria, we will also want to 
try to specify those subtle aspects of personality or personal style that may 
be relevant to qualities of performance. These may involve such\ traits as 
thoroughness, sensitivity to the emotional* state on the patient, personal 
stability, ethical makeup, and the like. ^ " - ^ 

Related perhaps to matters of personal style* are the more complex or inte- 
grative skills--rthe ability to define a proper practice in a given setting; 
for exajnple, or to liave a varied repertoire of responses' and the capability 
to use the most appropriate in a particular situation. 

We may frequently find it useful to consider the common errors or failures 
that, from our experience, we know are often made. The unused E K G apparatus 
rusting away, the one-time prescription of hormones, the high infant mortal- 
ity from untreated jaundice, and the like, may suggest components and dimen- 
sions that are quite critical to assess. 

A final consideratiop that I woj^ld like to place before you is that 
attention needs to be given not only to qualities of performance and to in- 
dices of productivity, but also to the consequences of particular behaviors 
by the physician. A physician who is active and diligent in informal com- 
munity health roles may err in diagnoses or be able to see fewer patients, but 
may achieve popular and (gffective support for an augmented health center.^ 

The ''test" we must build this time is thus remarkably different from the ■ 
UPJ^.tests of medical knowledge and the licensing examinations. We are not so 
much concerned with an inventory of factual knowledge which the physician . 
shoulli have but may or may not apply, as we are with what he 'does and how he 
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does it--with what benefits and with what deficiencies as compared with alter- 
native-activities. We shall be concerned' not so much with what answers should 
be g.iven to what questions, as we are with what'we can observe of the physician 
, and of his pat;ients, of his records, of the visible results of his practice. 

r \ 

What, then, are the many elenlents of quality of practice? What are the 
most effective salutiqns the individual physician can make to the health needs 
of the Island? What can he do toward the ultimate goal of increasing the prob- 
ability that training and other professional activities wilL improve their 
varied effectiveness of physicians in service? It is not only a matter of life 
and dea|h, but also a matter of achieving a maturity of the discipline of 
medical practice that few others have had the courage or the ability/ to carry 
very well. All our advisers tell us--as does the HEW financing of this con- 
ference--that you good people here just may be able to do it dn fine style. 
If so, the ramifications will go far beyond the shores of the Island.* 
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APPENDIX 

Adequate Utilization of Facilities and Resources 
(Department of Health Guidelines Adopted by Subgroup A) 
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In the evaluation of performance of a physician practicing in a health 
center, the conditions under which he practices should be taken into consid- 
eration. Facilities, equipment, financial resources 'available and para- 
medical personnel alThave a great influence on the quality of care he can 
render. 



The following items on the utilization of Facilities and Resources , 
can be used as a checklist (with yes o.r no answers) to help in the eval- 
uation of the physician's performance. 

A. Conditions of Physical Plant and Equipment^ 

1. Modem health center, with up-to-date equipment; with adequate space 
for all services 



2. Obsolete facility; lack of space, outmoded equipment with poor state 
of maintenance 

3. Conditions of physical plant and surroundings are sanitary, safe^'and 
attractive 

B. - Manpower Resources (Medical and Paramedical Personnel) / 

jf , 

' 1. Health center adequately staffed (2 employees per bed) 

2. Numbers of professional and technical personnel for/a 22-25 bed 
health center: , 

(a) Physicians — 4 ^ 

(b) Nurses 8 r 

ic) Medical technologist or Microscopist/(Lab . helper) — 1. 

(d) X-ray technician 1 • 

(e) Pharmacist or pharmacist's aide.^ * — 1 

(f) Medical record clerk --^ , 1 

(g) PraQtical nurses 16 

3. Personnel are well trained to do their jobs 

'4'. Responsibilities, wherever justified, are delegated by highly trained 
personnel to those with less specialized training 

5. Continuous in-service training available for the personnel 

6. Incentives available on the basis of individual merits - 

7. Morale is high, as judged by low absenteeism and low turnover 

C. Adequate Utilization of Facilities and Resources 

'l. .Patients are hospitalized who can be treated as outpatients 

2. ' Patients are hospitalized -who should have been referred to 

Regional Hospital - ^ ^' 

3. Only those patients who can benefit by admission to hpa'lth center 
are admitted 
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4. Patient's stay in health center is reasonable, according to his ^ 
' diagnosis, with an average of 3-4 days per patient 

5. Occupancy rate of health center is reasonable, within the following 
limits: 

Health centers with fewer than 25 beds 50760% 

Health cent'^rs between 26-50 bec^^ 60-70% 

Health centers over 50 beds - over 70^ occupancy 
6,. Admissions health center are approximately 1200 per year 
(22 bed health center) 

D. Financial Resources 

1. Appropriation for operation of health center is distributed on a 
12 -month ba|is 

2. Supplies are -purchased through bidding " 

3. Selection o£^:medical equipment is made in consultation with medical 
staff ^ f 

E. Laboratory anH^X-ray Services ^ 

1. Modern equipment available for laboratory tests to be performed at 
he alth^i center 

2. X-ray machine available, in good operating condition**'*^* 

3. Technicians :^available to operate equipment 

4. Physicians* ^requests for laboratory tests and x-rays are foljiowed 
promptly \. 

5. Ratios of laboratory tests and x-rays taken on patients admitted 
are within Acceptable ratios j(to be established) * * 

F. Pharmacy . ^ 

1. Pharmacy well-stocked for the needs of in-patiehts and outpatients 

2. Formulary adopted and in.use by medical staff 

3. A pharmacist in charge ^ . - - - 

4. A pharmacist's' aide in charge 

5. Drugs not indicated are not dispensed; e.g., novaldin, tetanus - 
antitoxin, sodium ajnytal fojL-Sj^izures 

Patients Records 




1. Record room' ample, well -locatccy, accessible to medical staff 

2. Records kept in an orderly manner 

3. A. trained person in charge of record room 

4. Medical records kept up to dat.e, completed, and according to 
P. A.S. standards 

Operating Room 

1. Well-equipped; used for minor surgery only in the 22 bed health 
center ' . . ^ . . . , , . 

2. Major surgery performed in health centers over SO beds, where a 
properly trained physician or a. surgeon is available. 
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